Rain structure transfer using an exemplar rain image for synthetic rain image generation

This letter proposes a simple method of transferring rain structures of a given exemplar image (rain image) into a target image (non‐rain image). Given the exemplar rain image and its corresponding masked rain image, rain patches including real‐life rain structures are extracted randomly, and then residual rain patches are obtained by subtracting those rain patches from their mean patches. Next, residual rain patches are selected randomly, and then added to the given target image along a raster scanning direction. To decrease boundary artifacts around the added patches on the target image, minimum error boundary cuts are found using dynamic programming, and then blending is conducted between overlapping patches. Our experiment shows that the proposed method can generate realistic rain images that have similar rain structures in the exemplar images. Moreover, it is expected that the proposed method can be used for rain removal. More specifically, non‐rain images and synthetic rain images generated via the proposed method can be used to learn classifiers (e.g., deep neural network) in a supervised manner. effectively [1]. Most computer vision algorithms depend on feature descriptors such as scale invariant feature transform (SIFT) [2] and histogram of oriented gradients (HOG) [3]. These descriptors are designed based on the gradient's magnitude and orientation, and thus rain structures can have negative effects on the feature extractor. For this reason, rain removal is a necessary tool [4]. However, to learn classifiers (e.g., deep neural network) in a supervised manner, it is necessary to collect rain and clean patch pairs. This requires synthetic rain image generation. There are various types of methods [5‐9] that